Image processing apparatus and method, and image processing program

ABSTRACT

Picture data is compressed and encoded in consideration of visual characteristics of human beings against colors. A color of pixels is detected corresponding to color difference components Cb and Cr for each sub block into which a macro block is divided. Each sub block is composed of four pixels×four lines. The number of pixels from which a predetermined color is detected is counted for each sub block. When the count value is ⅔ or greater of the total number of pixels of each sub block, it is determined that the macro block which contains the sub block contains the predetermined color. An activity is obtained using a luminance component Y for the macro block. The activity is multiplied by a quantizer priority value corresponding to the predetermined color. As a result, the activity is corrected. Corresponding to the corrected activity, the quantizer scale value is changed to a smaller value. With the changed quantizer scale value, the macro block is quantized with a smaller quantizer step value. As a result, the picture quality of a macro block from which a predetermined color has been detected is suppressed from being deteriorated.

TECHNICAL FIELD

The present invention relates to a picture processing apparatus, amethod thereof, and a picture processing program for compressing andencoding picture data in the unit of a block.

BACKGROUND ART

In a known picture data compressing and encoding system, picture data isquantized in the unit of a block composed of a predetermined number ofpixels. Such a compressing and encoding system has been used in forexample MPEG2 (Moving Pictures Experts Group 2). In the MPEG2, picturedata is processed corresponding to DCT (Discrete Cosine Transform) inthe unit of a block composed of a predetermined number of pixels.Obtained DCT coefficients are quantized. In such a manner, the picturedata is compressed and encoded. In the MPE2, a quantizer scale valuewhich represents a quantizer step is designated. With the designatedquantizer scale value, DCT coefficients are compressed and encoded.

In the compressing and encoding process corresponding to the MPEG2, anindex number referred to as an activity which represents the complexityand smoothness of a picture to be compressed is calculated and thepicture quality is optimized corresponding to an adaptive quantizingprocess based on the activity.

In this method, when a picture is simple and smooth, if it iscompressed, the deterioration of the picture quality is visuallyprominent. In this region (hereinafter referred to as a flat region),the picture is finely quantized with a quantizer scale value which is afine quantizer step. In contrast, when a picture is complicated, thedeterioration of the picture quality is not visually prominent. In thisregion, the picture is coarsely quantized with a quantizer scale valuewhich is a coarse quantizer step. In this method, with a limited codeamount, the picture quality is effectively optimized.

As was described above, when picture data is compressed, each pictureregion is divided into pixel blocks each of which has a predeterminedsize. Each block is quantized and processed corresponding to the DCTmethod. The MPEG2 standard prescribes a block of eight pixels×eightlines as the minimum process unit. Each block of eight pixels×eightlines is processed corresponding to the DCT method. DCT coefficientsobtained corresponding to the DCT method are quantized in the unit of amacro block of 16 pixels×16 lines.

On the other hand, the MPEG2 standard does not clearly prescribe a unitin which an activity is calculated. However, in TM5 (Test Model 5) ofthe MPEG2, it has been proposed that an activity should be processed inthe unit of a sub block of eight pixels and eight lines which is similarto a DCT block.

Next, an activity calculating method used in “Adaptive quantizationconsidering visual characteristics” adopted in the MPEG2 TM5 will bedescribed.

In the adaptive quantization, a quantizer scale value Qj is variedcorresponding to an activity of each macro block so that a code amountgenerated in one frame is controlled corresponding to the state of thepicture. As a result, the high picture quality is accomplished. Forexample, in a flat region of which the deterioration of the picturequality is visually prominent, each macro block is quantized with aquantizer scale value Qj which is a fine quantizing step. In acomplicated picture region of which the deterioration of the picturequality is relatively non-prominent, each macro block is quantized witha quantizer scale value Qj which is a coarse quantizer step. To do that,a quantizer scale value Qj is varied corresponding to an activity.

An activity is obtained by successively calculating the followingformulas (3), (2), and (1) for a j-th macro block using pixel values ofa luminance signal of an original picture for a total of eight blocks offour blocks of a frame DCT encoding mode and four blocks of a field DCTencoding mode rather than a prediction error.act _(j)=1+min[sblk=1, 8](var _(—) sblk)  (1)var _(—) sblk= 1/64Σ[k=1, 64](Pk−Pavg)²  (2)Pavg= 1/64Σ[k=1, 64]P _(k)  (3)

where Pk represents a pixel value in a luminance signal block of theoriginal picture. In the formula (3), pixel values of 64 pixels of ablock composed of eight pixels×eight lines are summed and divided by 6.As a result, an average value Pavg of the pixel values is obtained.Next, in the formula (2), the difference between the average value Pavgand Pk is obtained. As a result, average difference values of the blockcomposed of eight pixels×eight lines are calculated. In the formula (1),with the minimum value of the average difference values var_sblk, anactivity act_(j) of the macro block is obtained. This is because even ifa part of the macro block contains a flat portion, it is necessary tofinely quantize the macro block.

As visual characteristics of human beings against colors, thesensitivity of red is the highest. Thus, the deterioration of red can beeasily detected. Moreover, in a studio photography or the like, blueused in a blue back of a chroma-key is a color whose deteriorationshould be suppressed from a view point of a picture process.

Thus, it has been desired to suppress a compression ratio for a pictureregion containing a color component whose deterioration should besuppressed in color components which compose a color picture to becompressed and encoded so that colors can be represented more preciselythan before.

DISCLOSURE OF THE INVENTION

Thus, an object of the present invention is to provide a pictureprocessing apparatus, a method thereof, and a picture processing programfor suppressing a compression ratio of a region containing a particularcolor component so as to improve a visual effect in consideration ofvisual characteristics of human beings against colors.

To solve the forgoing problem, claim 1 of the present invention is apicture processing apparatus, comprising color detecting means fordetermining whether or not each block of picture data represents atleast one predetermined color; quantizing means for quantizing thepicture data with a quantizer step width which is set for each block;and quantizer step width changing means for setting a smaller quantizerstep width of the quantizer means to a block which represents thepredetermined color than a block which does not represents thepredetermine color.

In addition, claim 8 of the present invention is a picture processingmethod, comprising the steps of determining whether or not each block ofpicture data represents at least one predetermined color; quantizing thepicture data with a quantizer step width which is set for each block;and setting a smaller quantizer step width at the quantizer step to ablock which represents the predetermined color than a block which doesnot represents the predetermine color.

In addition, claim 15 of the present invention is a picture processingprogram for causing a computer apparatus to execute a picture processingmethod, comprising the steps of determining whether or not each block ofpicture data represents at least one predetermined color; quantizing thepicture data with a quantizer step width which is set for each block;and setting a smaller quantizer step width at the quantizer step to ablock which represents the predetermined color than a block which doesnot represents the predetermine color.

As was described above, according to the present invention, it isdetermined whether or not each block of picture data represents at leastone predetermined color. In addition, picture data is quantized with aquantizer step width which is set for each block. A smaller quantizerstop width is set to a block which represents a predetermined color thana block which does not represent the predetermined color. Thus, picturedata can be quantized in consideration of visual characteristics of eyesof human beings against colors.

BRIEF DESCRIPTION OF DRAWINGS

FIGS. 1A and 1B are block diagrams showing an example of the structureof a digital VTR according to an embodiment of the present invention;

FIGS. 2A, 2B, and 2C are block diagrams showing an example of thestructure of an MPEG encoder;

FIGS. 3A, 3B, and 3C are schematic diagrams showing an example of thestructure of a stream transferred in each portion of the MPEG encoder;

FIGS. 4A, 4B, and 4C are schematic diagrams showing an example of astream transferred in each portion of the MPEG encoder;

FIGS. 5A and 5B are schematic diagrams showing an example of thestructure of a stream transferred in each portion of the MPEG encoder;

FIGS. 6A and 6B are schematic diagrams showing an example of thestructure of a stream transferred in each portion of the MPEG encoder;

FIGS. 7A, 7B, and 7C are schematic diagrams showing an example of thestructure of a stream transferred in each portion of the MPEG encoder;

FIGS. 8A, 8B, and 8C are schematic diagrams showing an example of thestructure of a stream transferred in each portion of the MPEG encoder;

FIG. 9 is a schematic diagram showing an example of the structure of astream transferred in each portion of the MPEG encoder;

FIG. 10 is a block diagram showing an example of the structure of anactivity portion;

FIGS. 11A, 11B, and 11C are schematic diagrams for explaining a subblock forming method in an encoding mode;

FIG. 12 is a flow chart showing a process of the MPEG encoderaccomplished by software;

FIG. 13 is a graph showing a color difference plane (Cb, Cr) where colordifferences Cb and Cr are represented on a horizontal axis and avertical axis;

FIG. 14 is a block diagram showing an example of a hardware structurefor executing a color detecting process;

FIGS. 15A, 15B, 15C, and 15D are schematic diagrams for explaining therelation between the size of a sub block with which a color is detectedand a threshold value of a count value of pixels with which a color isdetected; and

FIGS. 16A, 16B, and 16C are schematic diagrams for explaining a subblock forming method in a chroma format.

BEST MODES FOR CARRYING OUT THE INVENTION

Next, an embodiment of the present invention will be described.According to the present invention, pixels of a predetermined color aredetected for each block of picture data. When the number of detectedpixels is equal to or greater than a predetermined threshold valueagainst the total number of pixels, it is determined that thepredetermined color has been detected from the block. A smallerquantizer scale value is set to a picture region which contains theblock than the other picture regions. With a smaller quantizer stepwidth, the picture region is more finely quantized. Consequently, thepicture quality of the picture region containing the predetermined coloris suppressed from being deteriorated and the visual effect for thecolor is improved.

According to the embodiment of the present invention, the value of anactivity, which represents the complexity of a picture, as was describedin the section of “Related Art,” is corrected corresponding to thedetected color. As a result, the quantizer step width of the pictureregion is varied.

An embodiment which will be described in the following is a preferredembodiment of the present invention. Although various technicallypreferred limitations are made in the following embodiment, unlessexplicitly stated that they restrict the present invention, the scope ofthe present invention is not restricted by such limitations.

Next, an embodiment of the present invention will be described. FIGS. 1Aand 1B show the structure of an example of a digital VTR according tothe embodiment of the present invention. The digital VTR can directlyrecord a digital video signal which has been compressed and encodedcorresponding to the MPEG system to a record medium.

First of all, the structure, process, and operation of a recordingsystem of the digital VTR will be described. Two types of serial-digitalinterface signals which are an SDI (Serial Data Interface) signal and anSDTI (Serial Data Transport Interface) signal and an external referencesignal REF as a control signal are input from the outside to therecording system.

The SDI is an interface prescribed by SMPTE. The SDI transmits a (4:2:2)component video signal, a digital audio signal, and additional data. TheSDTI is an interface which transmits an MPEG elementary stream which isa stream of which a digital video signal has been compressed and encodedcorresponding to the MPEG system (hereinafter this MPEG elementarystream is referred to as MPEG ES). An ES is composed of 4:2:2components. An ES is a steam composed of only I pictures and has therelation of 1 GOP=1 picture. In an SDTI-CP (Content Package) format, anMPEG ES is divided into access units. In addition, an MPEG ES is packedin packets in the unit of a frame. In the SDTI-CP, a sufficienttransmission band (at 27 MHz or 36 MHz of a clock rate or at 270 Mbps or360 Mbps of a stream bit rate) is used. In one frame period, an ES canbe transmitted as a burst.

An SDI signal transmitted by the SDI is input to an SDI input portion101. The SDI input portion 101 converts the input SDI signal as a serialsignal into a parallel signal and outputs the parallel signal. Inaddition, the SDI input portion 101 extracts an input synchronous signalwhich is a input phase reference from the SDI signal and outputs theextracted input synchronous signal to a timing generator TG 102.

In addition, the SDI input portion 101 separates the converted parallelsignal into a video signal and an audio signal. The separated videoinput signal and audio input signal are output to an MPEG encoder 103and a delay circuit 104, respectively.

The timing generator TG 102 extracts a reference synchronous signal fromthe input external reference signal REF. The timing generator TGgenerates a timing signal necessary for the digital VTR insynchronization with a designated reference signal which is one of thereference synchronous signal and the input synchronous signal which issupplied from the SDI input portion 101 and supplies the generatedtiming signal as a timing pulse to each block.

The MPEG encoder 103 performs the DCT process for the input video inputsignal so as to transform it into coefficient data. Thereafter, the MPEGencoder 103 quantizes the coefficient data. Thereafter, the MPEG encoder103 encodes the coefficient data with a variable length code (VLD). Thevariable length code encoded (VLC) data which is output from the MPEGencoder 103 is an elementary stream (ES) corresponding to the MPEG2. Theoutput is supplied to one input terminal of a recording sidemulti-format converter (hereinafter referred to as recording side MFC).

According to the embodiment of the present invention, as was describedabove, the MPEG encoder 103 performs quantization by changing aquantizer scale value corresponding to a color component inconsideration of visual characteristics of human beings against colors.

The delay circuit 104 functions as a delay line for matching the audioinput signal as non-compressed data with a delay of a process of theMPEG encoder 103 for the video signal. The audio signal delayed by thedelay circuit 104 for a predetermined amount is output to an ECC encoder107. This is because in the digital VTR according to the embodiment, theaudio signal is treated as a non-compressed signal.

The SDTI signal which is supplied from the outside and transmittedthrough the SDTI is input to an SDTI input portion 105. The SDTI inputportion 105 detects the synchronization of the SDTI signal. The SDTIsignal is temporarily stored in a buffer. An elementary stream isextracted from the SDTI signal.

The extracted elementary stream is supplied to the other input terminalof the recording side MFC 106.

The obtained synchronous signal is supplied to the forgoing timinggenerator TG 102 (not shown).

In addition, the SDTI input portion 105 extracts a digital audio signalfrom the input SDTI signal. The extracted digital audio signal issupplied to an ECC encoder 107.

In such a manner, the digital VTR according to the embodiment candirectly input an MPEG ES independent from a base band video signalwhich is input from the SDI input portion 101.

The recording side MFC circuit 106 has a stream converter and aselector. The recording side MFC circuit 106 selects an MPEG ES suppliedfrom the MPEG encoder 103 or an MPEG ES supplied from the SDTI inputportion 105 and groups DCT coefficients of a plurality of DCT blockswhich compose one macro block corresponding to each frequency component.The recording side MFC circuit 106 rearranges the DCT coefficients inthe order of lower frequency components. Hereinafter, a stream of whichcoefficients of an MPEG ES have been rearranged is referred to asconverted elementary stream. When an MPEG ES is rearranged, in a searchreproducing operation, as many DC coefficients and low order ACcoefficients as possible are obtained. As a result, they contribute tothe improvement of the quality of a searched picture. The convertedelementary stream is supplied to the ECC encoder 107.

A main memory (not shown) having a large storage capacity is connectedto the ECC encoder 107.

The ECC encoder 107 has a packing and shuffling portion, an audio outercode encoder, a video outer code encoder, an inner code encoder, anaudio shuffling portion, a video shuffling portion, and so forth asinternal devices. In addition, the ECC encoder 107 has a circuit whichassigns an ID to each sync block and a circuit which adds a synchronoussignal. According to a first embodiment of the present invention, as anerror correction code for a video signal and an audio signal, a productcode is used. The product code is composed of an outer code and an innercode. A video signal or an audio signal is encoded with an outer codeand an inner code in the vertical direction and horizontal direction,respectively, of a two-dimensional matrix thereof. As a result, a datasymbol is dually encoded. As an outer code and an inner code,Reed-Solomon code can be used.

The converted elementary stream which is output from the MFC circuit 106is supplied to the ECC encoder 107. In addition, audio signals which areoutput from the SDTI input portion 105 and the delay circuit 104 aresupplied to the ECC encoder 107. The ECC encoder 107 performs ashuffling process and an error correction code encoding process for thesupplied converted elementary stream and audio signals, adds an ID and asynchronous signal to each sync block, and outputs the resultant data asrecord data.

The record data which is output from the ECC encoder 107 is convertedinto a record RF signal by an equalizer EQ 108 which contains a recordamplifier. The record RF signal is supplied to a rotating drum 109 onwhich a rotating head is disposed in a predetermined manner. In reality,a plurality of magnetic heads having different azimuths for formingadjacent tracks are disposed on the rotating drum 109.

When necessary, a scrambling process may be performed for record data.When data is recorded, a digital modulation may be performed for thedata which is recorded. In addition, partial response class 4 andViterbi code may be used. The equalizer 108 contains both a recordingside structure and a reproducing side structure.

Next, the structure, process, and operation of the reproducing system ofthe digital VTR will be described. When data is reproduced from themagnetic tape 110, the data is reproduced as a reproduction signal bythe rotating drum 109. The reproduction signal is supplied to thereproducing side structure of the equalizer 108 which contains areproducing amplifier and so forth. The equalizer 108 equalizes thereproduction signal and shapes the waveform thereof. When necessary, theequalizer 108 performs a digital modulating operation and a Viterbidecoding operation. An output of the equalizer 108 is supplied to an ECCdecoder 111.

The ECC decoder 111 performs an inverse process of the forgoing ECCencoder 107. The ECC decoder 111 contains a large capacity main memory,an inner code decoder, an audio deshuffling portion, a video deshufflingportion, and an outer code decoder. In addition, for video data, the ECCdecoder 111 contains a deshuffling and depacking portion and a datainterpolating portion. Likewise, for audio data, the ECC decoder 111contains an audio AUX separating portion and a data interpolatingportion.

The ECC decoder 111 detects the synchronization of the reproductiondata, detects a synchronous signal added at the beginning of a syncblock, and extracts the sync block. Each sync block of the reproductiondata is error-corrected with an inner code. Thereafter, an IDinterpolating process is performed for each sync block. The reproductiondata for which the ID interpolating process has been performed isseparated into video data and audio data. A deshuffling process isperformed for each of the video data and the audio data. As a result,the sequence of data which was shuffled when the data was recorded isrestored to the original sequence. The deshuffled data iserror-corrected with an outer code.

When data has an error the ECC decoder 111 cannot correct due to itserror correcting capability, the ECC decoder 111 sets an error flag tothe data. With respect to error of video data, the ECC decoder 111outputs a signal ERR which represents data which contains an error.

Reproduction audio data which has been error-corrected is supplied to anSDTI output portion 115. In addition, the reproduction audio data issupplied to a delay circuit 114. The delay circuit 114 delays thereproduction time for a predetermined amount. The delayed reproductionaudio data is supplied to an SDI output portion 116. The delay circuit114 is disposed to absorb the delay of the video data in the process ofan MPEG decoder 113 which will be described later.

On the other hand, video data which has been error-corrected is suppliedas a reproduced converted elementary stream to a reproducing side MFCcircuit 112. The forgoing signal ERR is also supplied to the reproducingside MFC 112. The reproducing side MFC 112 performs the inverse processof the forgoing recording side MFC 106. The reproducing side MFC 112contains a stream converter. The stream converter performs the inverseprocess of the recording side stream converter. In other words, thestream converter rearranges DCT coefficients for each frequencycomponent to those for each DCT block. Thus, the reproduction signal isconverted into an elementary stream corresponding to the MPEG2. At thatpoint, when the signal ERR is supplied from the ECC decoder 111, thecorresponding data is substituted for a signal fully corresponding tothe MPEG2. The resultant signal is output.

The MPEG ES which is output from the reproducing side MFC circuit 112 issupplied to the MPEG decoder 113 and the SDTI output portion 115. TheMPEG decoder 113 decodes (restores) the supplied MPEG ES to the originalnon-compressed video signal. In other words, the MPEG decoder 113performs an inversely quantizing process and an inverse DCT process forthe supplied MPEG ES. The decoded video signal is supplied to the SDIoutput portion 116.

As was described above, audio data which has been separated from videodata by the ECC decoder 111 is supplied to the SDI output portion 116through the delay 114. The SDI output portion 116 maps the suppliedvideo data and audio data to an SDI format and converts them into an SDIsignal having a data structure of the SDI format. The SDI signal isoutput to the outside.

On the other hand, as was described above, audio data which had beenseparated from video data by the ECC decoder 111 has been supplied tothe SDTI output portion 115. The SDTI output portion 115 maps video dataand audio data as an elementary stream to an SDTI format and convertsthem into an SDTI signal having a data structure of an SDTI format. TheSDTI signal is output to the outside.

A system controller 117 (abbreviated as syscon 117 in FIGS. 1A and 1B)is composed of for example a microcomputer. The system controller 117communicates with each block using a signal SY_IO and controls theoverall operation of the digital VTR. When switches and so forthdisposed on an operation panel (not shown) are operated, correspondingcontrol signals are supplied to the system controller 117. Correspondingto the control signals, the system controller 117 controls operationssuch as a recording operation, a recording operation, and so forth ofthe digital VTR.

A servo 118 communicates with the syscon, system controller, 117 usingthe signal SY_SV and controls the traveling of the magnetic tape 110 andthe driving of the rotating drum 109 using the signal SV_IO.

FIGS. 2A, 2B, and 2C show the structure of an example of the forgoingMPEG encoder 103. Examples of the structure of a stream transferred ineach portion shown in FIG. 2A, FIG. 2B, and FIG. 2C are shown in FIG.3A, FIG. 3B, FIG. 3C, FIG. 4A, FIG. 4B, FIG. 4C, FIG. 5A, FIG. 5B, FIG.6A, FIG. 6B, FIG. 7A, FIG. 7B, FIG. 7C, FIG. 8A, FIG. 8B, FIG. 8C, andFIG. 9.

The MPEG encoder 103 is composed of an input field activity averageprocessing portion 103A, a pre-encode processing portion 103B, and anencoding portion 103C. The input field activity average processingportion 103A obtains an average value of activities of input video dataand supplies the obtained average value to the pre-encode processingportion 103B. With the average value of the activities, the pre-encodeprocessing portion 103B estimates a generated code amount of the inputvideo data which is quantized. Corresponding to the estimated result,while the encoding portion 103C is controlling the code amount, theencoding portion 103C actually quantizes the input video data, encodesthe quantized video data with a variable length code, and outputs theresultant data as an MPEG ES.

Corresponding to a horizontal synchronous signal HD, a verticalsynchronous signal VD, and a field synchronous signal FLD supplied fromthe timing generator TG 102 shown in FIGS. 1A and 1B, a timing generatorTG 220 generates a timing signal required for the MPEG encoder 103 andoutputs the generated timing signal thereto. A CPU I/F block 221 is aninterface with the system controller 117 shown in FIGS. 1A and 1B. Theoperation of the MPEG encoder 103 is controlled with a control signaland data exchanged through the CPU I/F block 221.

First of all, a process performed by the input field activity averageprocessing portion 103A will be described. Video data which is outputfrom the SDI input portion 101 and then input to the MPEG encoder 103 issupplied to an input portion 201. The video signal is converted into aninterface so that it is suitably stored in a main memory 203. Inaddition, a parity check is performed for the video data. The video datais output from the input portion 201 to a header creating portion 202.Using a vertical blanking region or the like, headers such assequence_header, quantizer_matrix, and gop_header of the MPEG arecreated. The created headers are stored in the main memory 203. Theseheaders are mainly designated by the CPU I/F block 221. In other thanthe vertical blanking region, the header creating portion 202 storesvideo data supplied from the input portion 201 to the main memory 203.

The main memory 203 is a frame memory for a picture. The main memory 203rearranges video data and absorbs a system delay. Video data isrearranged by for example an address controller (not shown) in such amanner that the address controller controls a read address of the mainmemory 203. In FIGS. 2A, 2B, and 2C, 8 lines, 0.5 frame, and 1 framedescribed in the block of the main memory 203 represent read timings ofthe main memory 203 as delay values. They are properly controlledcorresponding to a command issued by the timing generator TG 220.

A raster scan/block scan converting portion 204 extracts MPEG macroblocks from each line of the video data stored in the main memory 203and supplies the extracted MPEG macro blocks to an activity portion 205disposed downstream thereof. According to the embodiment of the presentinvention, since activities are calculated with only the first field,macro blocks which are output from the raster scan/block scan convertingportion 204 are composed of video data of the first field.

As with an example shown in FIG. 3A, a stream which is output as outputdata from the raster scan/block scan converting portion 204 containsvertical address information of the current macro block, horizontaladdress information thereof, a blank area having a predetermined size,and video data for the current macro block in the order.

The stream has a data length of 576 words each of which is composed ofeight bits. The last 512 words (referred to as data portion) is assignedas an area for storing video data for one macro block. The first 64words (referred to as header portion) contains the forgoing addressinformation of the current macro block and a blank area for data andflags embedded by each portion disposed downstream of the rasterscan/block scan converting portion 204.

A macro block corresponding to the MPEG is a matrix of 16 pixels×16lines. On the other hand, the input field activity average processingportion 103A performs a process for obtaining activities of only thefirst field. Thus, when up to eight lines of the first field are storedin the main memory 203, the process can be started. In reality, theprocess is started with a command issued by the timing generator TG 220.

The activity portion 205 calculates an activity of each macro block. TheMPEG encoder 103 calculates activities of only the first field. Thecalculated result is output as a field activity signal field_act. Thesignal field_act is supplied to an averaging portion 206. The averagingportion 206 cumulates activities of one field and obtains an averagevalue avg_act. The average value avg_act is supplied to an activityportion 209 of the pre-encode processing portion 103B which will bedescribed later. The activity portion 209 performs an activitynormalizing process for each macro block using the average value avg_actof the first field.

Thus, after the activity average value avg_act of the first field hasbeen obtained, with the average value, a pre-encoding process can beperformed while the quantizing process is adaptively performed.

Next, the pre-encode processing portion 103B will be described. A rasterscan/block scan converting portion 207A basically performs the sameprocess as the forgoing raster scan/block scan converting portion 204.However, since the raster scan/block scan converting portion 207Aperforms a pre-encoding process for estimating a code amount, the rasterscan/block scan converting portion 207A requires video data of both thefirst field and the second field. Thus, when up to eight lines of thesecond field are stored in the main memory 203, the raster scan/blockscan converting portion 207A can structure an MPEG macro block having asize of 16 pixels×16 lines. At that point, the process can be started.In reality, the process is started with a command issued by the timinggenerator TG 220.

Video data which is output from the raster scan/block scan convertingportion 207A is supplied to a DCT mode portion 208. The DCT mode portion208 selects a field DCT encoding mode or a frame DCT encoding mode to beperformed.

In this case, the encoding process is not actually performed. Instead,the sum of absolute values of difference values of pixels which areadjacent in the vertical direction is calculated in both the field DCTencoding mode and the frame DCT encoding mode. The sum obtained in theformer is compared with the sum obtained in the latter. The encodingmode in which the smaller value is obtained is selected. The selectedresult is temporarily inserted as DCT mode type data dct_typ which is aflag in the stream. The flag is supplied to the later stages. As shownin FIG. 3B, the DCT mode type data dct_type is stored on the end side ofthe blank area of the header portion.

The video data which is output from the DCT mode portion 208 is suppliedto a sub block forming portion (not shown). The sub block formingportion divides a macro block into sub blocks corresponding to theencoding mode decided by the DCT mode portion 208. The video data whichhas been divided into sub blocks is supplied to the activity portion209.

The activity portion 209 basically performs the same process as theforgoing activity portion 205. However, since the activity portion 209performs a pre-encoding process as was described above, using data ofboth the first field and the second field, the activity portion 209calculates an activity of each macro block. The activity portion 209obtains an activity act. As shown in FIG. 3C, the activity act is placedafter the macro block address of the header portion. Next, with theactivity act and the field activity average value avg_act obtained bythe averaging portion 206, the activity portion 209 obtains a normalizedactivity Nact.

As shown in FIG. 4A, the normalized activity Nact is temporarilyinserted as normalized activity data norm_act which is a flag into theheader portion of the stream. The flag is supplied to the downstreamstages. In the stream, the forgoing activity act is overwritten with thenormalized activity data norm_act.

On the other hand, the activity portion 209 detects color components ofpixels for each sub block and counts the number of pixels in a range ofwhich the detected color components are determined as a predeterminedcolor. When the counted result exceeds a predetermined threshold value,for example at least ½ of the total number of pixels which compose a subblock, it is determined that the predetermined color has been detectedin a macro block which contains the sub block. The normalized activitydata norm_act of the macro block is corrected to a smaller value.

An output of the activity portion 209 is supplied to a DCT portion 210A.The DCT portion 210A divides the supplied macro block into DCT blockseach of which is composed of eight pixels×eight lines. The DCT portion210A performs a two-dimensional DCT for each DCT block and generates DCTcoefficients. As shown in FIG. 4B, the DCT coefficients are placed inthe data portion of the stream and supplied to a quantizer table portion211A.

The quantizer table portion 211A quantizes DCT coefficients converted bythe DCT portion 210A with a quantizer matrix (quantizer_matrix). Asshown in FIG. 4C, DCT coefficients quantized by the quantizer tableportion 211A are placed in the data portion of the stream and thenoutput. An output of the quantizer table portion 211A is supplied to amulti-staged quantizing portion composed of a plurality of quantizingblocks 212, 212, . . . , a plurality of VLC portions 213, 213, . . . , aplurality of cumulating portions Σ 214, 214, . . . , and a plurality ofcumulating portions Σ 215, 215, . . . DCT coefficients quantized by thequantizer table portion 211A are quantized by the multi-stagedquantizing portion.

The quantizing blocks 211, 211, . . . quantize DCT coefficients withdifferent quantizer scale (quantizer_scale) values Q. The quantizerscale values Q have been prescribed in for example the standard of theMPEG2. Corresponding to the standard, each of the quantizing blocks 212,212, . . . is composed of n=31 quantizers. DCT coefficients arequantized with quantizer scales Qn assigned to the respective quantizingblocks 212, 212, . . . in a total of 31 steps. In the followingdescription, the quantizer scale values of the Q_n portions 212, 212, .. . are denoted by quantizer scale values Qn.

The quantizing blocks 212, 212, . . . quantize DCT coefficients withrespective quantizer scale values Qn. The quantized results of thequantizing blocks 212, 212, . . . are divided as integer parts andfractional parts and then output. As shown in FIG. 5A, the integer partsare placed as DCT coefficients for quantizer scale values Qn in the dataportion of the stream and supplied to the VLC portions 213, 213, . . . .The VLC portions 213, 213, . . . perform a scanning operation such as azigzag scanning operation for DCT coefficients for the respectivequantizer scales Qn, references a VLC table corresponding to thetwo-dimensional Huffman code, and encodes the resultant DCT coefficientswith a variable length code.

As shown in FIG. 5B, data which has been encoded with a variable lengthcode by the VLC portions 213, 213, . . . is placed in the data portionof the stream and then output. Outputs of the VLC portions 213, 213, . .. are supplied to the corresponding cumulating portions Σ 214, 214, . .. , respectively.

The cumulating portions Σ 214, 214, . . . each cumulate a generated codeamount of each macro block. When 31 types of quantizers are used, 31types of generated code amounts are obtained for each macro block. Asshown in FIG. 6A, the generated code amounts cumulated by the cumulatingportions Σ 214, 214, . . . are placed in the header portion of thestream. In other words, the generated code amounts quantized by thequantizing blocks 212, 212, . . . for each macro block are placed in theheader portion of the stream. The data portion of the stream is deleted.The stream in which the generated code amounts for each macro block havebeen embedded is supplied to the main memory 203 and then the encodingportion 103C which will be described later.

With the generated code amounts for each macro block which are outputfrom the cumulating portions Σ 214, 214, . . . , an adaptive quantizingprocess is performed. In other words, with the normalized activity datanorm_act in consideration of visual characteristics against colorsobtained by the forgoing activity portion 209, mqaunt as a quantizerscale in consideration of visual characteristics is obtained.mqaunt=Q _(—) n×norm_act  (4)

The cumulating portions Σ 215, 215, . . . each select a code amountcorresponding to a generated code amount for each macro block quantizedwith quantizer_scale (=mquant) in consideration of visualcharacteristics from generated code amounts for each macro blockobtained by the cumulating portions Σ 214, 214, . . . and cumulates thecode amounts for one frame.

The values cumulated for one frame with each quantizer scale value Qn bythe cumulating portions Σ 215, 215, . . . are generated code amounts(frame data rates) of the frame. They are a stream of n words as shownin FIG. 6B. The stream in which the frame data rates are embedded issupplied to a rate controlling portion 217 which will be describedlater. When 31 types of quantizers are used, 31 types of generated codeamounts are obtained for each frame.

Next, a method for obtaining a generated code amount will be describedin detail. For example, “a generated code amount of a frame by thecumulating portion Σ 215 corresponding to a Q_(—)4 portion 212” isobtained as follows.

For example, when

norm_act[1]=1.3

norm_act[2]=1.5

norm_act[3]=0.8

norm_act[4]=1.0

. . .

mqaunt[1]=4×1.3=5.2

: The generated code amount by the quantizing block 212 whose quantizerscale value Qn=5 is obtained from the header portion of the stream shownin FIG. 6A.

mqaunt[2]=4×1.5=6.0

: The generated code amount by the quantizing block 212 whose quantizerscale value Qn=6 is obtained from the header portion of the stream shownin FIG. 6A.

mqaunt[3]=4×0.8=3.2

: The generated code amount by the quantizing block 212 whose quantizerscale value Qn=3 is obtained from the header portion of the stream shownin FIG. 6A.

mqaunt[4]=4×1.0=4.0

: The generated code amount by the quantizing block 212 whose quantizerscale value Qn=4 is obtained from the header portion of the stream shownin FIG. 6A.

. . .

The generated code amounts for one frame are cumulated. This operationis performed by the cumulating portions Σ 215, 215, . . . correspondingto the Q_(—)1 portion 212 to Q_n portion 212. As a result, generatedcode amounts for one frame are obtained.

Next, the encode processing portion 103C will be described. The encodeprocessing portion 103C performs a final encoding process. As wasdescribed above, the pre-encode processing portion 103B estimatesgenerated code amounts for one frame in various encoding operations. Theencode processing portion 103C performs an encoding processcorresponding to the estimated generated code amounts for one frame sothat the generated code amount does not exceed a designated target codeamount and outputs the encoded data as an MPEG ES.

Although data used by the encode processing portion 103C has been storedin the main memory 203, as was described above, when generated codeamounts for one frame have been estimated in various encoding operationsof the pre-encode processing portion 103B, the encode processing portion103C can start the encoding process. As was described above, the processof each portion of the encode processing portion 103C is properlystarted corresponding to a command issued by the timing generator TG220.

A raster scan/block scan converting portion 207B processes video datawhich is read from the main memory 203 in the same manner as the rasterscan/block scan converting portion 207A and extracts a macro block of 16pixels×16 lines from the video data. As shown in FIG. 7A, the extractedmacro block is placed in a data portion corresponding to the headerportion shown in FIG. 6A and then supplied to a DCT mode portion 216.

As with the forgoing DCT mode portion 208, the DCT mode portion 216selects a field DCT encoding mode or a frame DCT encoding mode withwhich the encoding process is performed. At that point, the DCT modeportion 208 has selected an encoding mode. The selected result has beentemporarily inserted as DCT type data dct_typ into a stream (see FIG.7A). The DCT mode portion 216 detects the DCT type data dct_typ from thestream and switches the encoding mode between the field encoding modeand the frame encoding mode corresponding to the detected DCT type datadct_typ. An output of the DCT mode portion 216 is shown in FIG. 7B.

A macro block which is output from the DCT mode portion 216 is suppliedto a DCT portion 210B. As with the forgoing DCT portion 210A, the DCTportion 210B performs a two-dimensional DCT process for each DCT blockcomposed of eight pixels×eight lines. The resultant DCT coefficients areplaced in the data portion of the stream as shown in FIG. 7C and thenoutput from the DCT portion 210B.

A quantizer table portion 211B can be structured in the same manner asthe quantizer table portion 211A. DCT coefficients generated by the DCTportion 210B are quantized with a quantizer matrix. The DCT coefficientsquantized with the quantizer table portion 211B are placed in the dataportion of the stream as shown in FIG. 8A and then supplied to the ratecontrolling portion 217.

The rate controlling portion 217 selects a frame data rate from thosefor quantizer scale values Qn obtained by the cumulating portions Σ 215,215, . . . of the pre-encode processing portion 103B so that the framedata rate does not exceed the maximum generated code amount per framedesignated by the system controller 117 and is the closest thereto andobtains the quantizer scale value Q corresponding to the selected framedata rate. As shown in FIG. 8B, the quantizer scale value Q for eachmacro block is placed as qauntiser-scale on the rear end side of theheader portion of the stream.

The remaining code amount is obtained by the rate controlling portion217 with the maximum generated code amount (target code amount) perframe designated by the system controller 117 and supplied through theCPU I/F block 221 and the frame data rates for the quantizer scalevalues Qn supplied from the cumulating portions Σ 215, 215, . . . .

In addition, the quantizer scale (mquant) value for each macro block canbe decreased by one size in the range that it does not exceed thedifference between the maximum generated code amount per framedesignated by the system controller 117 and supplied through the CPU I/Fblock 221 and the generated code amount per frame of the frame data rateselected by the rate controlling portion 217. Thus, the maximumgenerated code amount per frame designated by the system controller 117and supplied through the CPU I/F block 221 can be nearly accomplished.As a result, a high picture quality can be accomplished.

The quantizing portion 218 extracts a quantizer scale (quantiases_scale)value designated in the forgoing manner by the rate controlling portion217 from the stream and quantizes DCT coefficients quantized by thequantizer table portion 211B corresponding to the extracted quantizerscale value. At that point, the quantizer scale value supplied from therate controlling portion 217 is a quantizer scale (mquant) valueobtained from the normalized activity data norm_act. As a result,adaptive quantization is performed in consideration of visualcharacteristics.

DCT coefficients quantized by the quantizing portion 218 are placed inthe data portion of the stream as shown in FIG. 8C and then supplied toa VLC portion 219. The DCT coefficients quantized and supplied to theVLC portion 219 are for example zigzag scanned. With reference to a VLCtable corresponding to two-dimensional Huffman code, the DCTcoefficients are encoded with a variable length code.

At that point, an MPEG header portion which contains MPEG headerinformation of the slice layer or lower layer as shown in FIG. 9 issubstituted for the header portion which composes the first half portionof the stream in the VLC portion 219 and then the resultant stream isoutput. A variable length code is placed in the data portion on thesecond half side of the stream. In addition, the variable length code isbit-shifted so that it is byte-assigned and output as an MPEG ES.

FIG. 10 shows an example of the structure of the activity portion 209.Color difference components Cb and Cr of video data which is output fromthe DCT mode portion 208 are supplied to a sub block forming portion300. According to the embodiment of the present invention, the sub blockforming portion 300 divides a macro block of 16 pixels×16 lines into subblocks of color difference data Cb and Cr each of which is composed offour pixels×four lines.

As one example, in the chroma format 4:2:2, as shown in FIG. 11A, amacro block contains color difference data Cb and Cr which are equal totwo DCT blocks each of which is composed of eight pixels×eight lines. Atotal of eight sub blocks which are four sub blocks for color differencedata Cr and four sub blocks for color difference data Cb are formed inone macro block. As another example, in the chroma format 4:4:4, a macroblock contains color difference data Cb and Cr which are equal to fourDCT blocks each of which is composed of eight pixels×eight lines. 16 subblocks of color difference data are formed in one macro block. In otherthan the chroma formats 4:2:2 and 4:4:4, sub blocks can be formed insuch a manner.

In FIGS. 11A, 11B, and 11C, fields to which lines belong aredistinguished with a hatched area and a non-hatched area.

When the DCT mode type data dct_type embedded in the stream by the DCTmode portion 208 represents that the encoding mode is the frame DCTencoding mode, as shown in FIG. 11B, one sub block contains data of twofields which compose one frame. In contrast, when the encoding mode isthe field encoding mode, as shown in FIG. 1C, one sub block is composedof data of the same field.

In the forgoing description, sub blocks are formed for only colordifference components Cb and Cr of video data. Of course, sub blocks canbe also formed for a luminance component Y of video data.

Data of the luminance component Y of video data which is output from thesub block forming portion 300 is supplied to an activity (Y) portion301. Data of the color difference components Cb and Cr is supplied to acolor detecting portion 302.

As was described above, the activity portion (Y) 301 obtains thenormalized activity Nact for each macro block using the suppliedluminance component Y of video data. The normalized activity Nact issupplied to an activity correcting portion 303. On the other hand, thecolor detecting portion 302 detects a predetermined color for each subblock corresponding to the supplied color difference data Cb and Cr ofvideo data. The color detecting portion 302 determines whether or notthe macro block contains the predetermined color corresponding to thedetected result. The determined result is supplied to the activitycorrecting portion 303.

The activity correcting portion 303 corrects the normalized activityNact supplied from the activity portion (Y) 301 corresponding to thedetermined result supplied from the color detecting portion 302. Thecorrected normalized activity Nact is embedded as normalized activitydata norm_act in the stream. The processes of the color detectingportion 302 and the activity correcting portion 303 will be described indetail.

FIG. 12 is a flow chart showing an example of the case that the processof the MPEG encoder 103 is performed by software. Since the processshown in FIG. 12 is the same as the process of the hardware, the processof the software will be briefly described corresponding to the processof the hardware.

In the process shown in FIG. 12, steps S1 to S7 correspond to theprocess of the forgoing input field activity average processing portion103A. Steps S11 to S21 correspond to the process of the forgoingpre-encode processing portion 103B. Steps S31 to S38 correspond to theprocess of the forgoing encode processing portion 103C.

At the first step S1, video data is captured through the inputinterface. At the next step S2, in the vertical blanking region, eachheader of the MPEG is created from the captured video data and stored ina memory. In other than vertical blanking region, the captured videodata is stored in the memory.

At step S3, the scanning mode for the video data is converted from araster scanning mode into a block scanning mode. A macro block isextracted from the video data. This operation is performed bycontrolling read addresses of the memory for the video data. At step S4,the activity of the first field of the macro block extracted from thevideo data is calculated. At step S5, the activity Activity (act) as thecalculated result is cumulated. The cumulated result as a cumulatedvalue sum is stored in the memory. A loop from step S3 to step S5 isrepeated until it is determined that the last macro block of the firstfield has been processed at step S6. In other words, the cumulated valuesum is the sum of activities of macro blocks for one field.

When the determined result at step S6 represents that up to the lastmacro block of one field has been processed, the flow advances to stepS7. At step S7, the cumulated value sum stored in the memory is dividedby the number of macro blocks of one field. As a result, the averagevalue Activity (avg_act) of the field activities of which activities ofone field are averaged is obtained. The obtained average value Activity(avg_act) is stored in the memory.

After the average value Activity (avg_act) of the field activities hasbeen obtained, the flow advances to step S11. As with step S3, at stepS11, the scanning mode of the video data stored in the memory isconverted from the raster scanning mode into the block scanning mode. Amacro block is extracted from the video data.

At step S12, the field DCT encoding mode or the frame DCT encoding modeis selected for the DCT process. The selected result is stored as DCTmode type data dct_typ to the memory. At step S13, the activity of amacro block for both the first and second fields is calculated. With theaverage value Activity (avg_act) of field activities, which was obtainedat step s7 and stored in the memory, a normalized activity Activity(norm_act) is obtained. The obtained normalized activity Activity(norm_act) is stored in the memory.

At step S14, a color of the macro block is detected. For example, apredetermined color is detected corresponding to color difference dataCb and Cr of video data which is read for each sub block from memory.Corresponding to the detected result, it is determine whether or not themacro block contains the predetermined color. At step S15, correspondingto the determined result at step S14, the normalized activity Activity(norm_act) of the macro block containing the predetermined color iscorrected. For example, the corrected normalized activity Activity (normact) is overwritten on the normalized activity Activity (norm_act) onthe memory.

At step S16, the macro block extracted from video data at step S11 isdivided into DCT blocks each of which is composed of eight pixels×eightlines. The DCT blocks are processed by two-dimensional DCT. At step S17,DCT coefficients into which the DCT blocks have been converted by thetwo-dimensional DCT are quantized with a quantizer table(quantizer_table). Thereafter, the flow advances to step S18.

A cycle from steps S18 to S21 is repeated for each quantizer scale(quantizer_scale) value Qn. As a result, processes equivalent to theforgoing Q_n portions 212, 212, . . . , VLC portions 213, 213,cumulating portions Σ 214, 214, . . . , and cumulating portions Σ 215,215, . . . are performed. In other words, at step S18, DCT coefficientsare quantized with the quantizer scale value Q=1.

After the DCT coefficients have been quantized at step S18, the flowadvances to step S19. At step S19, with reference to a VLC table, thequantized DCT coefficients are encoded with a variable length code. Atstep S20, the generated code amount of the macro block encoded with thevariable length code is calculated. The calculated generated code amountis stored in the memory. At step S21, it is determined whether or notthere is the next quantizer scale value Q. When the determined result atstep S21 represents that there is the next quantizer scale value (YES),the flow returns to step S18. At step S18, the process is performed forthe next quantizer scale value Q.

When the determined result at step S21 represents that the cumulatedvalues of the generated code amounts of the macro block have beenobtained for all the quantizer scale values Qn, the flow advances tostep S22. At step S22, the generated code amounts of the macro blockobtained at step S20 are cumulated for one frame. At that point, using avisual characteristic of the forgoing mquant, an adaptive quantizationis performed. At step S23, it is determined whether or not up to thelast macro block (MB) of one frame has been processed. When thedetermined result at step S23 represents that up to the last macro blockhas not been processed, the flow returns to step S11. In contrast, whenup to the last macro block has been processed and the generated codeamount for one frame has been estimated, the generated code amount forone frame in the adaptive quantization for each quantizer scale value Qnis stored in the memory. Thereafter, the flow advances to step S31. Atstep S31, a real encoding process is performed.

As with step S11, at step S31, the scanning mode for the video datastored in the memory is converted from the raster scanning mode into theblock scanning mode. As a result, a macro block is extracted from thevideo data. At the next step S32, the DCT encoding mode is designatedcorresponding to the DCT mode type data dct_typ stored in the memory atstep S12.

At step S33, the macro block which has been extracted from the videodata at step S31 is divided into DCT blocks each of which is composed ofeight pixels×eight lines. The two-dimensional DCT process is performedfor the DCT blocks. At step S34, DCT coefficients into which the DCTblocks have been converted by the two-dimensional DCT process arequantized with a quantizer table (quantizer_table). Thereafter, the flowadvances to step S35.

At step S35, corresponding to the generated code amount for one framefor each quantizer scale value Q estimated at steps S11 to S23, tocontrol a code amount generated in the real encoding process, aquantizer scale value used at step S36 is designated for each macroblock.

At the next step S36, using the quantizer scale value Q designated atstep S35, the DCT coefficients quantized with the quantizer table atstep S34 are quantized.

At step S37, the DCT coefficients quantized at step S36 are encoded witha variable length code with reference to a VLC table. At step S38, it isdetermined whether or not up to the last macro block of one frame hasbeen processed. When the determined result at step S38 represents thatup to the last macro block for one frame has not been processed (NO),the flow returns to step S31. At step S31, the quantizing process andthe variable length code encoding process are performed for the nextmacro block. In contrast, when the determined result at step S38represents that up to the last macro block for one frame has beenprocessed, the encoding process for one frame has been completed.

In the forgoing example, the pre-encoding process of steps S11 to S23and the encoding process of steps S31 to S38 are described asindependent processes. However, the present invention is not limited tosuch an example. For instance, data obtained by estimating a generatedcode amount at steps S11 to S23 is stored in the memory. Data which isobtained in a real encoding process is selected from the data stored inthe memory. As a result, the process of steps S31 to S38 can becontained as a loop in the process of steps S11 to S23.

Next, an activity correcting process corresponding to a detected coloraccording to the embodiment of the present invention will be described.The activity correcting process is performed by the activity portion 209shown in FIGS. 2A, 2B, and 2C or at steps S14 and S15 shown in FIG. 12.

When picture quality is optimized using adaptive quantization with anactivity in a compressing and encoding process corresponding to theMPEG2, since the deterioration of a flat portion of the picture isvisually prominent, the flat portion is finely quantized with a smallquantizer step using a small quantizer scale value. In contrast, sincethe deterioration of a complicated portion in the picture is notprominent, the complicated portion is coarsely quantized with a largequantizer step using a large quantizer scale value. In this method, thepicture quality can be effectively optimized in the range of a limitedcode amount.

However, in adaptive quantization with an activity, visualcharacteristics of human beings against colors of a picture have notbeen considered. For example, as visual characteristics of human beingsagainst colors, the sensitivity of red is high. Thus, the deteriorationof red can be easily detected. In addition, blue of a blue back used ina chroma-key is a color whose deterioration should be suppressed from aview point of a picture process.

Thus, according to the embodiment of the present invention, a colorwhose deterioration should be suppressed is detected. A macro blockcontaining the color is more finely quantized with a smaller quantizerstep. As a result, the deterioration of the picture quality issuppressed.

In reality, a color to be detected is set as a range on a colordifference plane represented by color differences Cb and Cr. It isdetermined whether or not color difference signals Cb and Cr of pixelsof each sub clock composed of four pixels×four lines are in the setrange of the color difference plane (Cb, Cr). The number of pixels inthe range is counted for each sub block. When the counted value exceedsa predetermined threshold value, it is determined that the color whichhad been set has been detected in the sub block. The predeterminedthreshold value is at least ½ or greater of the total number of pixelscontained in each sub block, for example ⅔ or greater thereof. In thiscase, when 11 or greater of 16 pixels contained in each sub blockcomposed of four pixels×four lines are counted, it is determined that acolor which had been set has been detected.

When the determined result represents that a color which had been sethas been detected in at least one of sub blocks contained in one macroblock, it is determined that the macro block contains the color whichhas been set.

In other words, in the chroma format 4:2:2, when a color which had beenset has been detected from at least one of eight sub blocks of one macroblock, it is determined that the macro block contains the color whichhad been set. Likewise, in the chroma format 4:4:4, when a color whichhad been set has been detected from one of 16 sub blocks contained inone macro block, it is determined that the macro block contains thecolor which had been set. This applies for other chroma formats besidesthe chroma formats 4:2:2 and 4:4:4.

Next, a method for setting a color to be detected will be described.FIG. 13 shows a color difference plane (Cb, Cr) of which colordifferences Cb and Cr are represented on a horizontal axis and avertical axis, respectively. Any color is represented as one point onthe color difference plane (Cb, Cr). For example, the positions of red(R), blue (B), and green (G) on the color difference plane (Cb, Cr) aredenoted by “+(plus)” marks (point R, point B, point G). According to theembodiment, a color to be detected is set as a range on the colordifference plane (Cb, Cr).

Red (R) as a color to be detected is set as a hatched range containingthe point R. In other words, red is set as a range surrounded by astraight line a, Cb=128, Cr=255, and Cr=176 on the color differenceplane (Cb, Cr). The straight line a is represented by the followingformula (5).Cr=−2Cb+384  (5)

Blue (B) as a color to be detected is set as a hatched range whichcontains the point B. In other words, blue is set as a range surroundedby a straight line a, Cr=128, Cb=176, and Cb=255 on the color differenceplane (Cb, Cr). The straight line a is represented by the followingformula (6).Cr=(−½)Cb+192  (6)

Green (G) as a color to be detected is set as a hatched range whichcontains the point G in FIG. 13. In other words, green is set as a rangesurrounded by Cr=32, Cr=0, Cb=48, and Cb=16 on the color differenceplane (Cb, Cr).

It should be noted that the forgoing ranges are just examples. Thus, thepresent invention is not limited to such ranges. In other words, as longas a color similar to the tone of a color to be detected can berepresented, the ranges can be set in another method. For example, inthe forgoing example, the ranges of red and blue are set with theinclined straight lines á and â. Alternatively, the range(s) of redand/or blue can be set as a rectangle like the range of green. In theforgoing example, ranges are surrounded by straight lines.Alternatively, ranges can be surrounded by curved lines. In the forgoingexample, colors to be detected are red, blue, and green. However, itshould be noted that the present invention is not limited to such anexample. In other words, any one or any two of red, blue, and green maybe set as a color(s) to be detected. Alternatively, any color(s) otherthan red, blue, and green may be set as a color(s) to be detected.

In the ranges which have been set in such a manner, a color differencelevel Cb[p] of the color difference Cb and a color difference levelCr[p] of the color difference Cr are compared for each pixel containedin each sub block. It is determined whether or not the color of eachpixel is in the range which has been set. The number of pixelsdetermined as the color in the range is counted for each sub block. Whenthe macro block contains at least one sub block of which the number ofpixels determined as the color in the range exceeds a predeterminedthreshold value, it is determined that the macro block contains thecolor which has been set. A quantizer priority value for the color whichhas been set is selected for the macro block. By multiplying anormalized activity Nact by the quantizer priority value, the normalizedactivity Nact is corrected.

Assuming that colors to be detected are red (R), blue (B), and green(G), the detecting method for each color will be described in detail.

First of all, the detecting method for red (R) will be described withreference to the following programming list. The following list isdescribed in C language, which has been used as a program describinglanguage for computer apparatuses and so forth. Number+“:” at thebeginning of each line represents a line number with which each line isdescribed.

1: R_det = 1.0; 2: for (n = 0; n < num_of_block; n++) { 3: R_cnt = 0; 4:for (p = 0; p < 16; p++) { 5: if ((Cr[p] >= −2*Cb[p] + 384) && (Cr[p] >=176) && (Cb[p] <= 128)) { 6: R_cnt++; 7: } 8: if (R_cnt >= R_num) { 9:R_det = R_scale; 10: break; 11: } 12: }

Line 1 represents that an initial value of a quantizer priority valueR_det for red is set to [1.0]. Line 2 represents that a processperformed in the following for each sub block is repeated for the numberof sub blocks contained in one macro block. The number of sub blockscontained in one macro block is given by a value num_of_block. In thechroma format 4:2:2, the value num_of_block is [8]. In the chroma format4:4:4, the value num_of_block is [16]. Line 3 represents that a countvalue R_cnt which represents the number of pixels determined as red isinitialized.

Line 4 represents that processes of lines 5 and 6 are repeated for thenumber of pixels of the current sub block. Line 5 represents the rangeof red (hatched range containing the point R) shown in FIG. 13. Inaddition, line 5 determines whether or not a p-th pixel p of the currentsub block is red. In other words, it is determined whether or not thecolor difference levels Cb[p] and Cr[p] of the pixel p satisfy thecondition given on line 5. When they satisfy the condition, it isdetermined that the pixel p is red. On line 5, a symbol “&&” representsa logical product “AND.” When the determined result represents that thepixel p is red, on line 6, the count value R_cnt is incremented by [1].

When the determination has been performed for 16 pixels of the currentsub block, on line 8 the count value R_cnt is compared with apredetermined parameter R_num. The parameter R_num is a threshold valuewhich represents the number of pixels in the total of pixels in thecurrent sub block with which the current sub block is determined as red.At that point, a value of at least ½ or greater of the total number ofpixels of the sub block is set as the parameter R_num so as to preventrandom noise from being mistakenly detected and detect a group of red.According to the embodiment, [12], which is ¾ of the total number ofpixels [16] of the sub block, is set as the parameter R_num.

When the determined result of the comparison of the count value R_cntand the parameter R_num on line 8 represents that the count value R_cntis equal to or greater than the parameter R_cnt, on line 9, apredetermined parameter R_scale is set as the quantizer priority valueR_det for red. In this example, the parameter R_scale is [0.5].

Line 10 represents that “break” causes the flow to exit from the loopfor which the number of repetitions has been set on line 2. In otherwords, when the count value R_cnt of at least one of sub blocks of onemacro block is equal to or greater than the parameter R_num, the processfor the macro block is finished. In addition, when the count value R_cntof each sub block matches the parameter R_num, the flow exits from theloop. Thereafter, the quantizer priority value R_det for red of themacro block is set as the value of the parameter R_scale.

When the quantizer priority value R_det is applied for the macro block,the quantizer scale value of the macro block is changed to a smallquantizer scale value which is a small quantizer step widthcorresponding to the quantizer priority value R_det. Thus, with thesmall quantizer step with, the macro block is more finely quantized. Inother words, the normalized activity Nact is multiplied by the quantizerpriority value R_det. The quantizer scale value is multiplied by thenormalized activity Nact. With the resultant quantizer scale valuemquant in consideration with the visual characteristics, the macro blockis quantized. Thus, in the example, the quantizer scale value mquant isaround half of that in the case that red is not detected. As a result,when the macro block is quantized, the generated code amount haspriority. Thus, the macro block is more finely quantized.

The range of red on the color difference plane (Cb, Cr) may be given bya rectangle as expressed as line 5′.

5′: if ((Cb[p]) >= R_Cb_lower) && (Cb[p] <= R_Cb_upper) && (Cr[p] >=R_Cr_lower) && (Cr[p] <= R_Cr_upper)    In this example, the parametersR_Cb_lower, R_Cb_upper, R_Cr_lower, and R_Cr_upper (see FIG. 13) aregiven as follows: R_Cb_lower = 64 R_Cb_upper = 128 R_Cr_lower = 176R_Cr_upper = 255

Next, the detecting method for blue (B) will be described with referenceto the following programming list. The following list is described in Clanguage, which has been used as a program describing language forcomputer apparatuses and so forth. Number+“:” at the beginning of eachline represents a line number with which each line is described.

1: B_det = 1.0; 2: for (n = 0; n < num_of_block; n++) { 3: B_cnt = 0; 4:for (p = 0; p < 16; p++) { 5: if ((Cr[p] >= (−½)*Cb[p] + 192) && (Cr[p]<= 128) && (Cb[p] >= 176)) { 6: B_cnt++; 7: } 8: if (B_cnt >= B_num) {9: B_det = B_scale; 10: break; 11: } 12: }

Line 1 represents that an initial value of a quantizer priority valueB_det for blue is set to [1.0]. Line 2 represents that a processperformed in the following for each sub block is repeated for the numberof sub blocks contained in one macro block. The number of sub blockscontained in one macro block is given by a value num_of_block. In thechroma format 4:2:2, the value num_of_block is [8]. In the chroma format4:4:4, the value num_of_block is [16]. Line 3 represents that a countvalue B_cnt which represents the number of pixels determined as blue isinitialized.

Line 4 represents that processes of lines 5 and 6 are repeated for thenumber of pixels of the current sub block. Line 5 represents the rangeof blue (hatched range containing the point B) shown in FIG. 13. Inaddition, line 5 determines whether or not a p-th pixel p of the currentsub block is blue. In other words, it is determined whether or not thecolor difference levels Cb[p] and Cr[p] of the pixel p satisfy thecondition given on line 5. When they satisfy the condition, it isdetermined that the pixel p is blue. When the determined resultrepresents that the pixel p is blue, on line 6, the count value B_cnt isincremented by [1].

When the determination has been performed for 16 pixels of the currentsub block, on line 8 the count value B_cnt is compared with apredetermined parameter B_num. The parameter B_num is a threshold valuewhich represents the number of pixels in the total of pixels in thecurrent sub block with which the current sub block is determined asblue. At that point, a value of at least ½ or greater of the totalnumber of pixels of the sub block is set as the parameter B_num so as toprevent random noise from being mistakenly detected and detect a blockof blue. According to the embodiment, [12], which is ¾ of the totalnumber of pixels [16] of the sub block, is set as the parameter B_num.

When the determined result of the comparison of the count value B_cntand the parameter B_num on line 8 represents that the count value B_cntis equal to or greater than the parameter B_cnt, on line 9, apredetermined parameter B_scale is set as the quantizer priority valueB_det for blue. In this example, the parameter B_scale is [0.75].

Line 10 represents that “break” causes the flow to exit from the loopfor which the number of repetitions has been set on line 2. In otherwords, when the count value B_cnt of at least one of sub blocks of onemacro block is equal to or greater than the parameter B_num, the processfor the macro block is finished. Thereafter, the quantizer priorityvalue B_det for blue of the macro block is set as the value of theparameter B_scale.

The quantizer priority value B_det is applied for the macro block in thesame manner as the detecting method for red. Thus, in the example, thequantizer scale value mquant is around 0.75 of that in the case thatblue is not detected. As a result, when the macro block is quantized,the generated code amount has priority. Thus, the macro block is morefinely quantized.

The range of blue on the color difference plane (Cb, Cr) may be given bya rectangle as expressed as line 5′.

5′: if ((Cb[p]) >= B_Cb_lower) && (Cb[p] <= B_Cb_upper) && (Cr[p] >=B_Cr_lower) && (Cr[p] <= B_Cr_upper)    In this example, the parametersB_Cb_lower, B_Cb_upper, B_Cr_lower, and B_Cr_upper which representranges on the color difference plane (Cb, Cr) (see FIG. 13) are given asfollows: B_Cb_lower = 176 B_Cb_upper = 255 B_Cr_lower = 64 B_Cr_upper =128

Next, the detecting method for green (G) will be described withreference to the following programming list. The following list isdescribed in C language, which has been used as a program describinglanguage for computer apparatuses and so forth. Number+“:” at thebeginning of each line represents a line number with which each line isdescribed.

1: G_det = 1.0; 2: for (n = 0; n < num_of_block; n++) { 3: G_cnt = 0; 4:for (p = 0; p < 16; p++) { 5: if ((Cb[p] >= G_Cb_lower) && (Cb[p] <=G_Cb_upper) && (Cr[p] >= G_Cr_lower) && (Cr[p] <= G_Cr_upper) { 6:G_cnt++; 7: } 8: if (G_cnt >= G_num) { 9: G_det = G_scale; 10: break;11: } 12: }

Line 1 represents that an initial value of a quantizer priority valueG_det for green is set to [1.0]. Line 2 represents that a processperformed in the following for each sub block is repeated for the numberof sub blocks contained in one macro block. The number of sub blockscontained in one macro block is given by a value num_of_block. In thechroma format 4:2:2, the value num_of_block is [8]. In the chroma format4:4:4, the value num_of_block is [16]. Line 3 represents that a countvalue G_cnt which represents the number of pixels determined as green isinitialized.

Line 4 represents that processes of lines 5 and 6 are repeated for thenumber of pixels of the current sub block. Line 5 represents the rangeof green (hatched range containing the point G) shown in FIG. 13. Inaddition, line 5 determines whether or not a p-th pixel p of the currentsub block is green. In other words, it is determined whether or not thecolor difference levels Cb[p] and Cr[p] of the pixel p satisfy thecondition given on line 5. When they satisfy the condition, it isdetermined that the pixel p is green. When the determined resultrepresents that the pixel p is green, on line 6, the count value G_cntis incremented by [1].

In this example, the parameters G_Cb_lower, G_Cb_upper, G_Cr_lower, andG_Cr_upper which represent ranges on the color difference plane (Cb, Cr)(see FIG. 13) are given as follows:

-   G_Cb_lower=16-   G_Cb upper=48-   G_Cr_lower=0-   G_Cr_upper=32

When the determination has been performed for 16 pixels of the currentsub block, on line 8 the count value G_cnt is compared with apredetermined parameter G_num. The parameter G_num is a threshold valuewhich represents the number of pixels in the total of pixels in thecurrent sub block with which the current sub block is determined asgreen. At that point, a value of at least ½ or greater of the totalnumber of pixels of the sub block is set as the parameter G_num so as toprevent random noise from being mistakenly detected and detect a blockof green. According to the embodiment, [12], which is ¾ of the totalnumber of pixels [16] of the sub block, is set as the parameter G_num.

When the determined result of the comparison of the count value G_cntand the parameter G_num on line 8 represents that the count value G_cntis equal to or greater than the parameter G_cnt, on line 9, apredetermined parameter G_scale is set as the quantizer priority valueG_det for green. In this example, the parameter G_scale is [1]. In thiscase, although green is detected, the quantizer scale value mquant isnot changed against the case that green is not detected. Thus, when themacro block is quantized, the generated code amount does not havepriority.

Line 10 represents that “break” causes the flow to exit from the loopfor which the number of repetitions has been set on line 2. In otherwords, when the count value G_cnt of at least one of sub blocks of onemacro block is equal to or greater than the parameter G_num, the processfor the macro block is finished. Thereafter, the quantizer priorityvalue G_det for green of the macro block is set as the value of theparameter G_scale.

Next, a final quantizer priority value deciding method in the case thatquantizer priority values of a plurality of colors have been detectedfrom the same macro block will be described. When quantizer priorityvalues R_det, B_det, and G_det of red, blue, and green are detected fromthe same macro block, the minimum value is selected from those quantizerpriority values. The selected value is treated as the quantizer priorityvalue for the macro block. This process can be expressed as follows in Clanguage.Color_scale=min(R_det, B_det, G_det)

In other words, a final quantizer priority value Color_scale of themacro block is decided by an operator min for selecting the minimumvalue from the quantizer priority values R_det, B_det, and G_det ofthese colors. With the decided quantizer priority value Color_scale, theactivity portion (Y) 301 changes the normalized activity Nact obtainedfrom the luminance component Y. As a result, the detected result of eachcolor is reflected to a quantizer scale value. Assuming that thenormalized activity Nact is normalized activity data norm_act, thecorrection of the normalized activity Nact with the quantizer priorityvalue Color_scale can be represented as following in C language.norm_act=Color_scale×norm_act

FIG. 14 shows an example of a hardware structure for executing theforgoing color detecting process. The structure corresponds to the colordetecting portion 302 and the activity correcting portion 303 shown inFIG. 10. The structure shown in FIG. 14 accomplishes the forgoing colordetecting process and normalized activity Nact correcting process. Thestructure for detecting each of blue and green is almost the same as thestructure for detecting red. Thus, for simplicity, FIG. 14 shows thestructure for detecting red, omitting the structure for detecting blueand green.

Color difference data Cr of a pixel of a sub block is input to an inputterminal a of a comparator 33, an input terminal a of a comparator 35,and an input terminal b of a comparator 36. Color difference data Cb ofthe same pixel is input to an input terminal b of a comparator 34, aninput terminal b of a comparator 37, and a one-bit shift circuit 31.

The color difference data Cb which is input to the one-bit shift circuit31 is shifted by one bit and thereby the value of the color differencedata Cb is doubled. The resultant color difference data Cb is input to afirst input terminal of a subtracting device 32. A value [384] is inputto a second input terminal of the subtracting device 32. The subtractingdevice 32 subtracts an output of the one-bit shift circuit 31 from thevalue [384]. An output of the subtracting device 32 is input to an inputterminal b of the comparator 33. The comparator 33 compares values whichare input to the input terminals a and b. When the color difference dataCr is equal to or greater than the output of the subtracting device 32,the comparator 33 outputs a “H” level, for example a value [1]. Thecompared result of the comparator 33 is input to a first input terminalof an AND circuit 38.

A value [128] is input to an input terminal a of the comparator 34. Thecomparator 34 compares the value [128] and the color difference data Cbwhich is input to the input terminal b. When the color difference dataCb is equal to or smaller than the value [128], the comparator 34outputs a “H” level. A value [176] is input to an input terminal b ofthe comparator 35. The comparator 35 compares the value [176] and thecolor difference data Cr which is input to the input terminal a. Whenthe color difference data Cr is equal to or greater than the value[176], the comparator 35 outputs a “H” level. The compared results ofthe comparators 34 and 35 are input to second and third input terminalsof the AND circuit 38, respectively.

The AND circuit 38 calculates a logical product of the compared resultsof the comparators 33, 34, and 35 which are input to the first, second,and third input terminals of the AND circuit 38. When all the inputs area “H” level, the AND circuit 38 outputs a “H” level. In other words, theone-bit shift circuit 31, the subtracting device 32, and the comparator33 determines the straight line a for the pixel. A logical product ofthe determined result and the compared results of the comparators 34 and35 is calculated so as to determine whether or not the color of thepixel is in the hatched range containing the point R shown in FIG. 13.When the determined result represents that the color of the pixel is inthe range, the AND circuit 38 outputs an “H” level.

Alternatively, in the circuit shown in FIG. 14, it is determined whetheror not the color of the pixel is in a rectangle on the color differenceplane (Cb, Cr). In this case, an upper limit value R_Cr_upper and alower limit value R_Cr_lower of the color difference Cr for red areinput to the input terminals a and c of the comparator 36. When thecolor difference data Cr is in the ranges of the upper limit valueR_Cr_upper and the lower limit value R_Cr_lower, the comparator 36outputs a “H” level. Likewise, an upper limit value R_Cb_upper and alower limit value R_Cb_lower of the color difference Cb for red areinput to the input terminals a and c of the comparator 37. When thecolor difference data Cb is in the ranges of the upper limit valueR_Cb_upper and the lower limit value R_Cb_lower, the comparator 37outputs a “H” level. The outputs of the comparators 36 and 37 are inputto first and second input terminals of an AND circuit 39. The ANDcircuit 39 calculates a logical product of the outputs of thecomparators 36 and 37. Corresponding to the result, it is determinedwhether or not the color of the pixel is in the ranges of the upperlimit value R_Cr_upper and the lower limit value R_Cr_lower of the colordifference Cr for red and in the ranges of the upper limit valueR_Cb_upper and the lower limit value R_Cb_lower of the color differenceCb for red. When the color of the pixel is in the ranges, the ANDcircuit 39 outputs a “H” level.

The outputs of the AND circuit 38 and AND circuit 39 are input to aselector 40. The selector 40 selects one of the inputs corresponding toa selection signal Rsel. The selection signal Rsel is a signal suppliedfrom the system controller 117 corresponding to user's setting or thelike.

An output of the selector 40 is input to a register 41. A value which isread from the register 41 is input to a first input terminal of anadding device 42. An output of the adding device 42 is input to aregister 43. A value which is read from the register 43 is input to asecond input terminal of the adding device 42. In addition, a valuestored in the register 43 is reset for each sub block. In other words,the adding device 42 and the register 43 cumulate the output of theselector 40 for each sub block and calculate a count value R_cnt for subblocks.

The count value R_cnt which is read from the register 43 is supplied toa comparator 44. An output of the comparator 44 is input to a register45. The comparator 44 compares the count value R_cnt of sub blocksstored in the register 45 and the count value R_cnt which is output fromthe register 43 and updates the register 45 with the larger valuethereof. As a result, the maximum value of the count values R_cnt forred of sub blocks of one macro block is obtained. An output of theregister 45 is input to an input terminal a of a comparator 46. Thecomparator 46 compares the output of the register 45 and a parameterR_num for a threshold value for the number of pixels detected as red.The parameter R_num has been input from an input terminal b of thecomparator 46. The compared result of the comparator 46 is supplied as aselection signal to a selector 47.

A parameter R_scale for a quantizer priority value R_det is input to afirst input terminal of the selector 47. A fixed value [1.0] is input toa second input terminal of the selector 47. The parameter R_scale is forexample a value [0.5]. The selector 47 selects one of the first andsecond input terminals corresponding to the selection signal suppliedfrom the comparator 46. In other words, when the count value R_cnt isequal to or greater than the parameter R_num, the comparator 46 selectsthe first input terminal and outputs the parameter R_scale. Theparameter R_scale or fixed value [1.0] selected by the selector 47 isoutput as a quantizer priority value R_det for red. The quantizerpriority value R_det is input to a first input terminal of a comparator82 through a register 48.

On the other hand, blue and green are detected in the same manner asred. As a result, a quantizer priority value B_det for blue and aquantizer priority value for green are obtained. The quantizer priorityvalue B_det for blue and the quantizer priority value G_det for greenare input to a second input terminal and a third input terminal of thecomparator 82.

According to the embodiment of the present invention, green is detectedin a rectangular range on the color difference plane (Cb, Cr) shown inFIG. 13. Thus, the structure for the one-bit shift circuit 31, thesubtracting device 32, the comparators 33, 34, and 35, the AND circuit38, and the selector 40 shown in FIG. 14 is omitted.

The comparator 82 compares the quantizer priority values R_det, B_det,and G_det for red, blue, and green and outputs the minimum value of thequantizer priority values R_det, B_det, and G_det as a final quantizerpriority value_Color_scale. The final quantizer priority valueColor_scale is input to a first input terminal of a multiplying device84 through a register 83.

Normalized activity data norm_act obtained from the luminance componentY of the macro block by the activity portion (Y) 301 is input to asecond input terminal of the multiplying device 84. The multiplyingdevice 84 multiplies the normalized activity data norm_act which isinput to the second input terminal by the quantizer priority valueColor_scale which is input to the first input terminal. Thus, thenormalized activity data norm_act is corrected with the quantizerpriority value Color_scale and the corrected normalized activity datanorm_act is output.

When a plurality of colors red, blue, and green have been detected froma macro block, the minimum value of those quantizer priority values isselected as a final quantizer priority value Color_scale. With theselected quantizer priority value Color_scale, the original normalizedactivity norm_act is changed. As a result, the quantizer scale value isdecreased. With the decreased quantizer scale value, the macro block ismore finely quantized. In this state, data is compressed and encoded.When no color has been detected, the quantizer priority value for eachcolor is set to a fixed value [1.0]. Thus, the normalized activity Nactis not changed. As a result, with the original normalized activity Nact,the macro block is quantized.

Next, a method for setting a threshold value for the number of pixelsdetected as a color will be described. When it is determined whether ornot a predetermined color (for example, red) has been detected from eachblock by counting the number of pixels from which red has been detectedfor each block, the color may be incorrectly detected due to noise.

For example, it is assumed that a threshold value with which red isdetected for each block composed of eight pixels×eight lines like eachDCT block is ½ or greater than the total number of pixels of the block.FIGS. 15A and 15B show an example of which red has been detected from ½of the total number of pixels of a block composed of eight pixels×eightlines. In FIGS. 15A and 15B, the count value R_cnt is [32], which is ½of the total number of pixels [64] of the block. In FIGS. 15A, 15B, 15C,and 15D, shaded squares represent pixels detected as red.

When a block has 32 red pixels which are adjacent and successive asshown in FIG. 15B, they are recognized as a group of red by human eyes.In this case, red tends to be easily recognized and thereby thedeterioration thereof tends to be visually detected. In contrast, whenred pixels disperse in a block at random, human eyes hardly recognizethem as red. In this case, the deterioration is not easily detected.Thus, in the case shown in FIG. 15A, it may be not necessary to decreasethe quantizer priority value to a value smaller than [1], decrease thequantizer scale value, and finely quantize each block. In addition, asshown in FIG. 15A, when red pixels disperse at random, they may beaffected by noise.

To detect a group of red, the threshold value for the count value ofpixels detected as red is strictly set to for example ¾ or greater. FIG.15C shows an example of which red is detected with a threshold valuewhich is ¾ of the total number of pixels of a block. In the exampleshown in FIG. 15C, the count value R_cnt is [48], which is ¾ of thetotal number of pixels [64] of the block. When the threshold value isstrictly set, red pixels do not disperse at random. As a result, a groupof red pixels appears.

However, when the threshold value is increased, as shown in FIG. 15C,unless the area of a block of red is greater than that shown in FIG.15B, red cannot be detected from a block.

When the compression rate is low, the code amount is relativelyaffordable. In this case, since the picture quality of the entirepicture becomes high, the deterioration of the picture quality of amacro block which locally contains a predetermined color, for example aboundary portion of a predetermined color, becomes more prominent. Thus,when a red area in a block is as shown in FIG. 15B, it is necessary tocorrectly detect it, decrease the quantizer scale value for the block,and finely quantize the block so as to suppress the picture quality frombeing deteriorated.

On the other hand, if a block in which red pixels randomly disperse dueto for example noise as shown in FIG. 15A is finely quantized with adecreased quantizer scale value, even if the block does not require highpicture quality, it uses a code amount. Thus, since the code amountlimited for each frame cannot be optimally used, the picture qualitycannot be optimized.

To solve such a problem, according to the embodiment of the presentinvention, the size of each block from which a color is detected isdecreased. For example, each block is composed of four pixels×fourlines. Like a block shown in FIG. 15C, FIG. 15D shows an example ofwhich a color is detected with a threshold value which is ¾ of the totalnumber of pixels of each block composed of four pixels×four lines. Whenthe count value R_cnt of pixels detected as red is [12], which is thethreshold value, the pixels detected as red do not randomly disperse.Instead, as shown in FIG. 15D, a group of red pixels appears. Inaddition, the area of a group of pixels detected as red does notvisually become large as shown in FIG. 15C.

When at least one block from which a predetermined color has beendetected is contained in a macro block which is quantized with the samequantizer scale value, it is determined that the macro block containsthe predetermined color. Thus, according to the present invention, anactivity of a macro block which locally contains a predetermined color(namely, a so-called edge portion, for example a contour portion of acharacter or a picture) can be suppressed. Thus, with a smallerquantizer step width, the macro block can be finely quantized. As aresult, high picture quality can be accomplished.

In the above description, it is assumed that in the chroma format 4:2:2,one macro block contains eight sub blocks each of which is composed offour pixels×four lines. In reality, as shown in FIG. 16B, in the chromaformat 4:2:2, a sub block of four pixels×four lines is an oblong block.Thus, as shown in FIG. 16C, each sub block may be a square blockcomposed of two pixels×four lines. As a result, each macro block may beprocessed as 16 sub blocks. In the chroma format 4:4:4, as shown in FIG.16A, a sub block of four pixels×four lines is a square block. One macroblock contains 16 sub blocks.

As was described above, the color detecting process and quantizer scalevalue changing process corresponding to a detected color correspondingto the embodiment of the present invention can be accomplished by any ofhardware and software. A program which causes for example a computerapparatus to execute those processes is recorded on a CD-ROM (CompactDiscRead Only Memory) and provided therewith. When the CD-ROM is loadedto a CD-ROM drive of the computer apparatus and the programs recorded onthe CD-ROM are installed to the computer apparatus, the processes can beexecuted on the computer apparatus. Since the structure of the computerapparatus is well known, the description thereof will be omitted.

In the forgoing example, as an embodiment, the present invention isapplied for a digital VTR. However, the present invention is not limitedto that. For example, the present invention can be applied for aquantizer calculating circuit which has a color detecting function and aquantizer scale value changing function corresponding to a detectedcolor.

In addition, the sizes of the forgoing macro block and sub block arejust examples. In other words, the present invention can be applied forany combination of other sizes of blocks.

In addition, in the forgoing description, the threshold value of thecount value of pixels detected as each color in each sub block is thesame. In addition, the size of each sub block is the same. However, thepresent invention is not limited to those. For example, a thresholdvalue which differs in each color to be detected can be used. Likewise,each color can be detected for each sub block whose size differs in eachcolor.

In addition, in the forgoing description, the threshold value of thenumber of sub blocks determined that a macro block contains apredetermined color is 1. The present invention is not limited to suchan example. Instead, the threshold value may be 2 or greater. In otherwords, when a predetermined color has been detected from a plurality ofsub blocks (for example, two sub blocks) of a macro block, it can bedetermined that the macro block contains the predetermined color.

As was described above, when an activity representing the complexity ofeach macro block of a picture is decided, a color component contained ineach macro block is detected. When a predetermined color component hasbeen detected from a macro block, the activity thereof is changed sothat the macro block is more finely quantized. Thus, a picture can becompressed and encoded in consideration of visual characteristics ofhuman beings against colors.

In addition, according to the present invention, a color componentcontained in a macro block is detected for each sub block which issmaller than each macro block and which is composed of for example fourpixels×four lines. In addition, when a predetermined color has beendetected from pixels which are equal to or greater than a thresholdvalue of ⅔ or greater of the total number of pixels of the sub block, itis determined that the predetermined color is contained in all the macroblock which contains the sub block.

Thus, a color can be detected from a macro block which locally containsa predetermined color (namely, a so-called edge portion such as acontour portion of a character or a picture).

Thus, the detection size can be optimized. In addition, only a group ofcolor can be detected without an influence of noise.

1. A picture processing apparatus, comprising: sub block forming meansfor dividing each of a plurality of main blocks that are the minimumprocess blocks of picture data into a plurality of sub blocks; colordetecting means for detecting whether each of the plurality of mainblocks represents a predetermined color by detecting whether at leastone sub block contained by the corresponding main block represents apredetermined color, wherein the sub block's pixels that have a valuebeing related with the predetermined color and being within apredetermined range are counted and when the count exceeds apredetermined value, the sub block is detected to represent thepredetermined color; quantizing means for quantizing the picture datawith a quantizer step width that is set for each main block; andquantizer step width changing means for setting a smaller quantizer stepwidth of the quantizing means to a main block that represents thepredetermined color than a main block which does not represent thepredetermined color.
 2. The picture processing apparatus as set forth inclaim 1, further comprising: converting means for performing apredetermined conversion for each main block of the picture data beforethe picture data is quantized by the quantizing means.
 3. The pictureprocessing apparatus as set forth in claim 2, wherein the predeterminedconversion is DCT.
 4. The picture processing apparatus as set forth inclaim 1, wherein when the determined result of the color detecting meansrepresents that at least ½ of pixels of a sub block represent thepredetermined color, the color detecting means determines that the subblock represents the predetermined color.
 5. The picture processingapparatus as set forth in claim 1, wherein the quantizer step widthchanging means changes an activity based on a detected result of thecolor detecting means, the activity being a value corresponding to thecomplexity of each main block of a picture and being used to set aquantizer scale value, so as to set a smaller quantizer step width ofthe quantizer means to a main block which represents the predeterminedcolor than a main block which does not represent the predeterminedcolor.
 6. A picture processing method, comprising the steps of: dividingeach of a plurality of main blocks that are minimum process blocks ofpicture data into a plurality of sub blocks; detecting whether each ofthe plurality of main blocks represents a predetermined color bydetecting whether at least one sub block contained by the correspondingmain block represents a predetermined color, wherein the sub block'spixels that have a value being related with the predetermined color andbeing within a predetermined range are counted and when the countexceeds a predetermined value, the sub block is detected to representthe predetermined color; quantizing the picture data with a quantizerstep width that is set for each main block; and setting a smallerquantizer step width at the quantizer step to a main block thatrepresents the predetermined color than a main block which does notrepresent the predetermined color.
 7. The picture processing method asset forth in claim 6, further comprising the step of: performing apredetermined conversion for each main block of the picture data beforethe picture data is quantized at the quantizing step.
 8. The pictureprocessing method as set forth in claim 7, wherein the predeterminedconversion is DCT.
 9. The picture processing method as set forth inclaim 6, wherein when the determined result at the color detecting steprepresents that at least ½ of pixels of a sub block represent thepredetermined color, the color detecting step is performed bydetermining that the sub block represents the predetermined color. 10.The picture processing method as set forth in claim 6, wherein thequantizer step width changing step is performed by changing an activitycorresponding to a detected result at the color detecting step, theactivity being a value corresponding to the complexity of each mainblock of a picture and being used to set a quantizer scale value, so asto set a smaller quantizer step width at the quantizer step to a mainblock which represents the predetermined color than a main block whichdoes not represent the predetermined color.
 11. A computer-readablemedium storing a computer program for causing a computer apparatus toexecute a picture processing method, comprising the steps of: dividingeach of a plurality of main blocks that are the minimum process blocksof picture data into a plurality of sub blocks; detecting whether eachof the plurality of main blocks represents a predetermined color bydetecting whether at least one sub block contained by the correspondingmain block represents a predetermined color, wherein the sub block'spixels that have a value being related with the predetermined color andbeing within a predetermined range are counted and when the countexceeds a predetermined value, the sub block is detected to representthe predetermined color; quantizing the picture data with a quantizerstep width that is set for each main block; and setting a smallerquantizer step width at the quantizer step to a main block thatrepresents the predetermined color than a main block which does notrepresent the predetermined color.
 12. A picture processing apparatus,comprising: a sub block forming device configured to divide each of aplurality of main blocks that are the minimum process blocks of picturedata into a plurality of sub blocks; a color detecting device configuredto detect whether each of the plurality of main blocks represents apredetermined color by detecting whether at least one sub blockcontained by the corresponding main block represents a predeterminedcolor, wherein the sub block's pixels that have a value being relatedwith the predetermined color and being within a predetermined range arecounted and when the count exceeds a predetermined value, the sub blockis detected to represent the predetermined color; a quantizing deviceconfigured to quantize the picture data with a quantizer step width thatis set for each main block; and a quantizer step width changing deviceconfigured to set a smaller quantizer step width of the quantizingdevice to a main block that represents the predetermined color than amain block which does not represent the predetermined color.
 13. Thepicture processing apparatus of claim 12, wherein the sub block is 4pixels by four lines.
 14. The picture processing apparatus of claim 12,wherein the predetermined color is red color.
 15. The picture processingapparatus of claim 12, wherein the main block is a DCT block.
 16. Thepicture processing apparatus of claim 12, wherein the picture data isencoded to the MPEG standard.
 17. The picture processing apparatus ofclaim 12, wherein the quantizer step width changing device determines aquantizer priority value for the main block.
 18. The picture processingapparatus of claim 1, wherein the sub block is 4 pixels by four lines.19. The picture processing apparatus of claim 1, wherein thepredetermined color is red color.
 20. The picture processing apparatusof claim 1, wherein the main block is a DCT block.
 21. The pictureprocessing apparatus of claim 1, wherein the picture data is encoded tothe MPEG standard.
 22. The picture processing apparatus of claim 1,wherein the quantizer step width changing means determines a quantizerpriority value for the main block.